The present invention relates to a computer controllable display system and in particular to the interaction of a user with a computer controlled displayed image.
Computer controlled projection systems generally include a computer system for generating image data and a projector for projecting the image data onto a projection screen. Typically, the computer controlled projection system is used to allow a presenter to project presentations that were created with the computer system onto a larger screen so that more than one viewer can easily see the presentation. Often, the presenter interacts with the projected image by pointing to notable areas on the projected image with his/her finger, laser pointer, or some other pointing device or instrument.
The problem with this type of system is that if a user wants to cause any change to the projected image he/she must interact with the computer system (not the projected image) using an input device such as a mouse, keyboard or remote device. For instance, a device is often employed by a presenter to remotely control the computer system via infrared signals to display the next slide in a presentation. However, this can be distracting to the viewers of the presentation since the presenter is no longer interacting with them and the projected presentation and, instead, is interacting with the computer system. Often, this interaction can lead to significant interruptions in the presentation.
Hence, a variation of the above system has been developed to overcome the computer-only interaction problem to allow the presenter to directly interact with the projected image and thus give the presenter better interaction with the audience. In this system, the computer generates image data (e.g. presentation slides) to be projected onto a projection screen with an image projector. The system also includes a digital image capture device such as a digital camera for capturing the projected image. The captured projected image data is transmitted back to the computing system and is used to determine the location of any objects (e.g., pointing device) in front of the screen. The computer system may then be controlled dependent on the determined location of the pointing device. For instance, in U.S. Pat. No. 5,138,304 assigned to the assignee of the subject application, a light beam is projected onto the screen and is detected by a camera. To determine the position of the light beam, the captured image data of the projected image and the original image data are compared to determine the pointer position. The computer is then caused to position a cursor in the video image at the pointer position or is caused to modify the projected image data in response to the pointer position.
In order to implement a user interactive, computer controlled display or projection system, it is necessary to first identify the location of the displayed or projected image within the captured image data and then analyze the captured data corresponding to the displayed or projected image to identify and locate objects in front of the image. The object location information can then be used by the computer system to respond to the location of the objects to modify the displayed or projected image as desired by the presenter.
In this type of system, it is also desirable for the system to additionally be able to identify and locate the point of interest of the object. For instance, the point of interest of the pointer would be the tip of a pointer or the tip of a presenter""s finger. Hence, the presenter can then accurately point to a specific location within the displayed image so as to cause the system to respond dependent on that specific location of the point of interest.
Previously, the identification and location of the point of interest of an object in front of the display has been achieved by performing a detailed image analysis of the captured image data requiring exhaustive mathematical manipulations.
Hence, what is needed is a technique of locating the point of interest of an object located in front of a displayed or projected image in a user interactive, computer controlled display system simple enough to be performed during real-time operation.
A system and method of identifying a point of interest of an object located in front of a computer controlled display area captured by an image capture device is described. In accordance with the system and method, the point of interest of an object that extends into the captured display area is the point of the object that is the furthest distance from the peripheral boundary of the display area that has the shortest path of adjacent pixels of the captured display area data to the boundary.
One embodiment of the method is performed by capturing an image of the display area to obtain captured data, identifying pixels within the captured data that correspond to the object within the captured data, identifying pixels within the captured data that correspond to the peripheral boundary of the display area within the captured data, identifying and storing a subset of pixels common to the object pixels and the peripheral boundary pixels, and then performing a search using the common pixels and the object pixels to locate a pixel corresponding to the point of interest of the object. In one embodiment, the search is performed using a breadth-first search algorithm.
One embodiment of the system for identifying a point of interest of an object located in front of a computer controlled display area captured by an image capture device includes a computing system for displaying at least one image in a display area in response to image data, an image capture device for capturing image data including the display area within a capture area, means for identifying pixels in the captured data corresponding to the object in the display area, means for identifying pixels in the captured data corresponding to the peripheral boundary of the display area, means for identifying and storing the subset of pixels common to the object pixels and the boundary pixels, and means for searching the common pixels and the object pixels to determine a pixel corresponding to the point of interest of the object in the display area. In one embodiment, the means for searching comprises a breadth-first searching algorithm implementable in one of software, hardware or a combination of software and hardware.